Methods and systems for identifying the crossing of a virtual barrier

ABSTRACT

Systems, methods and media are disclosed for identifying the crossing of a virtual barrier. A person in a 3D image of a room may be circumscribed by a bounding box. The position of the bounding box may be monitored over time, relative to the virtual barrier. If the bounding box touches or crosses the virtual barrier, an alert may be sent to the person being monitored, a caregiver or a clinician.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S.Nonprovisional application Ser. No. 15/857,696, titled “Methods andSystems for Identifying the Crossing of a Virtual Barrier”, filed Dec.29, 2017, the entirety of which is hereby incorporated by reference.

TECHNICAL FIELD

This disclosure relates generally to computerized methods and systemsfor identifying the crossing of a virtual barrier.

BACKGROUND

Patients in institutional, home-like or in-home care are often monitoredfor a variety of behaviors. One concern is the possibility of a patientfalling, for example, while trying to get out of bed without assistance.Falls can cause significant harm to the patient, interfere with medicalcare for other issues, and increase burdens on caregivers or medicalproviders for the patient. Another concern is a patient touching,removing or otherwise disturbing medical equipment. Self-adjustment orremoval of nasogastric tubes or intravenous lines, for example, cancause harm to the patient, impede medical treatment, and add to theheavy burdens of a caregiver or medical practitioner who must restorethe medical equipment. For children or patients with dementia, it may bedesirable to know if a patient is leaving a room or other space withouta chaperone who can insure the patient's safety. It would be desirableto assist caregivers and medical providers with the monitoring for thesekinds of events, to help improve patient safety and reduce the burden oncaregivers and medical providers.

BRIEF SUMMARY

This brief summary is meant to present an overview of concepts relatedto this disclosure, and is expressly not meant to define or identify keyelements of the disclosure in isolation from the remainder of thedisclosure, including the figures.

This disclosure generally relates to systems, methods, and media foridentifying the crossing of a virtual barrier. The systems and methodsmay be computerized and, once configured, may operate without humanengagement or intervention unless or until an alarm condition arises.

In some aspects, this disclosure relates to a method of identifying thecrossing of a virtual barrier. The method comprises receiving image datafor a room from one or more 3D motion sensors. The method comprisesconfiguring a virtual barrier within the room, by a computerizedmonitoring system. The method comprises classifying an object within theroom as a person. The method comprises creating a bounding box tocircumscribe at least a portion of the person. The method comprisesmonitoring a position of the bounding box over time and relative to thevirtual barrier. The virtual barrier may be configured automatically bythe computerized monitoring system. The virtual barrier may beconfigured by the computerized monitoring system based on input from ahuman user. The method may further comprise sending an alert to acomputerized communication system if the bounding box touches or crossesthe virtual barrier. Upon receiving the alert, the computerizedcommunication system notifies the person, a caregiver, or clinician thatthe virtual barrier has been crossed. An alert may be sent to thecomputerized communication system only if a specified minimum portion ofthe bounding box crosses the virtual barrier. The image data may be sentto a centralized monitoring station. The centralized monitoring stationmay display human-intelligible images from the image data.

In some aspects, this disclosure relates to a system for identifying thecrossing of a virtual barrier. The system comprises a computerizedmonitoring system in communication with one or more 3D motion sensorsand a computerized communication system. The computerized monitoringsystem is configured to receive image data from the one or more 3Dmotion sensors, configure a virtual barrier, analyze the image data tocreate a bounding box around a person in the image data, and send analert to the computerized communication system if the bounding boxtouches or crosses the virtual barrier. The system may further comprisea user input device. The computerized monitoring system may beconfigured to configure the virtual barrier based on user input. Thesystem may further comprise a centralized monitoring station configuredto display human-intelligible images based on the data from the one ormore 3D motion sensors. The centralized monitoring station may beconfigured to receive an alert from the computerized communicationsystem and, upon receipt of an alert, move the display ofhuman-intelligible images to an alert display. The virtual barrier maybe configured automatically by the computerized monitoring system. Upondetecting that the bounding box has touched or crossed the virtualbarrier, the computerized monitoring system may be configured to confirmthe crossing of the virtual barrier by skeletal tracking, blob tracking,or combinations thereof. The computerized monitoring system may beconfigured to use biometrics to identify the person.

In some aspects, this disclosure relates to non-transitorycomputer-readable media having embodied thereon instructions which, whenexecuted by a computer processor, cause the processor to receive imagedata for a room from one or more 3D motion sensors, configure a virtualbarrier within the room by a computerized monitoring system, classify anobject within the room as a person, create a bounding box tocircumscribe at least a portion of the person, and monitor a position ofthe bounding box over time and relative to the virtual barrier. Theinstructions may further cause the processor to accept user input toconfigure the virtual barrier. The instructions may further cause theprocessor to send an alert to a computerized communication system if thebounding box touches or crosses the virtual barrier. The instructionsmay further cause the processor to send another alert to thecomputerized communication system if all of the bounding box crosses thevirtual barrier. The instructions may further cause the processor tosend the image data to a centralized monitoring station.

The claimed invention may improve the function of the computer processorin analogous systems for monitoring patients by deploying new analyticalmodels which require less processing speed and/or memory capacity thanprior systems, while still allowing for the use of 3D data that canimprove the accuracy of detection and reduce false alarms.

Additional objects, advantages, and novel features of this disclosurewill be set forth in part in the description which follows, and in partwill become apparent to those skilled in the art upon examination of thefollowing, or may be learned by practice of this disclosure.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

This disclosure references the attached drawing figures, wherein:

FIG. 1 is a schematic overview of an exemplary system and method foridentifying the crossing of a virtual barrier, in accordance withaspects of this disclosure;

FIG. 2 is a schematic overview of an exemplary system and method forcentralized monitoring, in accordance with aspects of this disclosure;

FIG. 3 is a view of an exemplary centralized monitoring primary display,in accordance with aspects of this disclosure;

FIG. 4 is a view of an exemplary centralized monitoring primary display,in accordance with aspects of this disclosure;

FIG. 5 is a view of an exemplary centralized monitoring primary display,in accordance with aspects of this disclosure;

FIG. 6 is a view of an exemplary centralized monitoring primary display,in accordance with aspects of this disclosure;

FIG. 7 is a view of an exemplary centralized monitoring primary display,in accordance with aspects of this disclosure;

FIG. 8 is a view of an exemplary centralized monitoring primary display,in accordance with aspects of this disclosure;

FIG. 9 is a view of an exemplary centralized monitoring primary display,in accordance with aspects of this disclosure; and

FIG. 10 is a simplified schematic view of an exemplary computingenvironment useful in practicing some aspects of this disclosure.

DETAILED DESCRIPTION

The subject matter of the present invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

As used in this disclosure, a “patient” refers to a person beingmonitored, without regard to whether that person is under the immediatecare of a medical professional, e.g., in a hospital, clinic, surgicalcenter, outpatient diagnostic center, outpatient treatment center,rehabilitation facility, hospice care facility, assisted livingfacility, group home, private home, or other environment. As used inthis disclosure, a “caregiver” may be a clinician, such as a doctor,nurse, physician's assistant, nurse's aide, orderly, physical therapist,and the like, or may be a paid or unpaid assistant who helps the patientwith health care and/or tasks of daily living, such as a visiting nurse,a home health aide, a paid companion, a relative, a friend, or the like.

Patient movement may be monitored for a variety of reasons. Patients whoare ill, disabled, or disoriented (e.g., because of injury or illness oras a side-effect of medication) may be more prone to fall than otherpeople. Monitoring a fall-prone patient's movements can assistcaregivers in providing prompt care to prevent a fall. For example,knowing when a patient is attempting to get out of bed would allow acaretaker to report quickly to the patient's bed and assist the patientto prevent a fall. Patients may, intentionally or unintentionally,disturb or damage medical equipment. For example, an uncomfortable oragitated patient may dislodge a nasogastric tube, or rub or pick at anintravenous line. These contacts can introduce germs from the patient'shands, displace medical equipment, damage medical equipment, and evencause personal harm. Knowing when a patient reaches for or ismanipulating medical equipment may allow for early intervention, so thatthe equipment can be maintained as needed for the patient's safety, oradjusted in a manner that is safe for the patient and will not harm theequipment. As another example, it may be desirable to monitor whenothers approach a patient's bedside. For example, it may be desirable toevaluate the time that a caregiver spends with a patient, and whetherthe time in close proximity to the patient is consistent with certainmedical needs, such as repositioning the patient as part of a bedsoreprevention regimen.

Having a caregiver monitor a patient is not practical. Mostcaregivers—whether in a medical institution or in a home-caresetting—have other responsibilities and cannot directly observe thepatient 24 hours a day. Some monitoring systems may monitor the patientbased on computer modeling of the patient and the patient'ssurroundings, such as the patient's bed. These systems aredata-intensive, requiring in-room computer processing equipment orsignificant bandwidth to communicate data to maintain the 3D model overtime. Also, systems which rely on modeling the patient and objects inthe room with the patient, such as the patient's bed, may fail undercommon conditions, such as low-lighting, a patient covered bybedclothes, objects placed between the camera and the patient, etc.

The methods, systems, and computer-readable media disclosed hereinprovide for automated, computerized systems that can help identify thecrossing of a virtual barrier. The virtual barrier can be flexibly setby a caregiver or other system user, to adapt to a variety of monitoringneeds. The virtual barrier can also be set so as to detect behaviorlikely to result in an undesired outcome—such as a patient moving in away that suggests the patient is trying to get out of bed—to allow timefor a caregiver to intervene. The patient may be monitored usingprobability-based bounding boxes, which allow for image analysis thatcan identify a person or people in an image without further modeling,thereby reducing the data and processing capacity requirements foron-going monitoring. Alternately, probability-based bounding boxes canbe used with more elaborate modeling, either as a back-up in case theprimary modeling mode fails (e.g., because low lighting or obstructionsdeny the system data needed to update the model) or as a primarydetection mode that can be verified against more elaborate modeling if apotential problem is detected.

An exemplary method and system for identifying the crossing of a virtualbarrier are shown in FIG. 1. A 3D motion sensor 120 may be co-locatedwith patient 110, who is tended by caregiver 100. For example, 3D motionsensor 120 may be located in or near a hospital room, bedroom, or otherlocation where patient 110 spends a significant amount of time. The 3Dmotion sensor 120 may be positioned to have a view of most or all of thepatient's body.

In general, the 3D motion sensor 120 is an electronic device thatcontains one or more cameras capable of identifying individual objects,people, and motion, regardless of lighting conditions. The 3D motionsensor 120 may further comprise one or more microphones to detect audio.As used in this disclosure, unless expressly described otherwise,reference to a sensor or sensors encompasses the singular and theplural, e.g., a singular sensor or an array of sensors, and an array ofsensors may be physically housed in a unitary structure or may bephysically distinct devices. The cameras may utilize technologiesincluding, but not limited to, color RGB, CMOS sensors, infraredprojectors, RF-modulated light, Time of Flight (ToF, including LIDAR),and combinations thereof. The 3D motion sensor may further contain oneor more microprocessors and/or image sensors to detect and processinformation sent and/or received by the one or more cameras. Suitable 3Dmotion sensors can perceive depth, in contrast to 2D cameras whichperceive only lateral and longitudinal positions. Exemplary 3D motionsensors include the Microsoft® Kinect® Camera, the Sony® Playstation®Camera, the Intel® RealSense® Camera, the Orbbec® Astra® Camera, theOrbbec® Persee® Camera, and the Asus® Xtion® Camera. Some of theseexemplary 3D motion sensors include one or more microphones, althoughmany aspects of the disclosure can be practiced without sensing audio.The 3D motion sensor may produce a depth map which is derived from ToFor comparable spatial analysis of the room, rather than pixel-basedimage analysis.

The 3D motion sensor 120 may be in electronic communication with acomputerized monitoring system 130, either as a separate component ofthe same physical unit or device, or as separate devices. The 3D motionsensor 120 may be co-located with or remote from computerized monitoringsystem 130, so long as data can be sent by the 3D motion sensor 120 tothe computer monitoring system 130 or retrieved by the computerizedmonitoring system 130 from the 3D motion sensor 120.

The 3D motion sensor 120 may operate continuously, or intermittently(for example, running for a fixed period at defined intervals), or on atrigger (e.g., when a motion detector or light sensor is activated,suggesting activity in the room). The 3D motion sensor 120 may operatecontinuously at all times while the monitoring is occurring, regardlessof whether the person or object of interest is moving or not. The 3Dmotion sensor 120 may view the entire room or a large portion of theroom by placement in a manner sufficient for the room to be visible tothe camera. Alternately, the 3D motion sensor 120 may view any portionof the room that includes the patient or a portion of the patient to bemonitored. The 3D motion sensor 120 may record video, or may forwardvideo to the computerized monitoring system 130 or directly to adatabase, such as database 170, for storage. Video is a series ofsequential, individual picture frames (e.g., 30 frames per second ofvideo). The video data may include 3D depth data, data defining one ormore bounding boxes, skeletal object tracking data and/or blob or objecttracking data. In some implementations, it may be desirable for thesensors to capture video only, or sound only, or video and sound. Videoonly (with 3D depth data, bounding box data. skeletal object trackingdata and/or blob or object tracking data) may make monitored patientsmore comfortable having conversations with visitors or caregivers thanif sound is also captured. Alternatively, or additionally, to protectpatient privacy and modesty, video displays of the image data from the3D motion sensor may be blurred or pixelated or otherwise obscured, orthe people and objects in the room may be converted from detailed imagedata to cartoons, less detailed drawings, or stick figures whendisplayed. The 3D motion sensor may collect and transmit data sufficientfor measuring and analyzing movement and interaction between differentpeople within the room, but transmit only sufficient image data for apartially obscured video, or a microprocessor associated with the 3Dmotion sensor and/or computerized monitoring station may process imageand/or video data to make the individuals and/or details of the room orthe activity of the room more difficult to distinctly identify. In someaspects, only 3D depth data, bounding box data, skeletal object trackingdata and/or blob or object tracking data is transmitted, without videoor still images.

With skeletal tracking alone, there can be factors affecting thecameras/image-video quality which affects the ability of thedetection/monitoring system to detect a skeleton. Such factors,especially in a hospital, include, but are not limited to,sheets/blankets covering a patient, trays positioned over the bed hidingthe patient and the patient blending into the bed and not having askeleton recognized.

With blob detection alone, there can be an increase in false positivesin detecting falls and “at risk” behavior. These false positives canoccur because blob detection does not differentiate between types of 3Dobjects. Blankets, treys, caretakers, or other 3D objects can trigger anautomated notification. Blob recognition also does not differentiatebetween parts of the body.

With bounding box detection, it may be difficult to fine-tune settings,such as confidence levels and thresholds for determining whether aboundary has been crossed, to achieve a balance between catching earlymovement toward a boundary and reducing false alarms, e.g., from apatient reaching for an object on a nearby table or nightstand. However,bounding box detection may require less data processing and/or may becomputed more quickly than skeletal or blob detection.

Combinations of skeletal tracking, blob detection, and bounding boxdetection can overcome some of the drawbacks of using any individualapproach. If a skeletal tracking system is unable to track a skeleton,then a virtual blob detection system and/or bounding box detectionsystem may be used to capture and/or recognize movement. If a virtualblob detection system and/or bounding box detection system hasinsufficient specificity to avoid false alarms at a rate unacceptable toa given user, skeletal tracking can be selectively used to confirm orclarify determinations made initially by virtual blob detection and/orbounding box detection. All three systems could be run simultaneously,or one system could be run preferentially to the other two, or onesystem could be run routinely with one or both of the other systems runto confirm potential detections.

The computerized monitoring system 130 may receive and analyze data from3D motion sensor 120. The computerized monitoring system 130 and/or the3D motion sensor 120 may be configured to monitor and/or analyze only aportion of the full view of the 3D motion sensor 120. For example, 3Dmotion sensor might be capable of viewing most or all of a room, or aroom and part of an adjacent hallway. However, to reduce processingcapacity and communication bandwidth requirements, the 3D motion sensor120 may be configured to capture data from a limited view, and/or thecomputerized monitoring system 130 may be configured to analyze only aportion of the data from 3D motion sensor 120. For example, thecomputerized monitoring system 130 may analyze data from a pre-definedarea around a patient, or around a patient's bed or chair.

Computerized monitoring system 130 is specifically designed andprogrammed to monitor activity based on information received from 3Dmotion sensor 120. Computerized monitoring system 130 may use imageclassification (e.g., identification of an “object” as a person), facialrecognition, height, distance between body points, and/or otherbiometrics (e.g., iris scanning, fingerprints, etc.) to “lock onto” thepatient for analysis, helping to avoid the possibility of thecomputerized monitoring system 130 tracking a visitor or caregiver whomight pass between the patient and the 3D motion sensor, or others whomay enter the 3D motion sensor's field of view. Computerized monitoringsystem 130 may use facial recognition, height, distance between bodypoints, etc. to identify one or more caregivers for the patient,distinct from the features of the patient. Alternately, or in addition,3D motion sensors and/or additional sensors, such as an RFID reader, mayread an electronic receiver, transmitter, or transceiver associated withthe patient and/or with a caregiver to identify and/or distinguishindividuals in the room. The patient and/or the caregiver may wear,carry, or otherwise be associated with such a transceiver in the form ofa badge, token, bracelet, cell phone, or other device. As one example,the patient may wear, carry, or otherwise be associated with atransmitter and the caregiver may wear, carry, or otherwise beassociated with a receiver. Alternately, the patient may wear, carry, orotherwise be associated with a receiver and the caregiver may wear,carry, or otherwise be associated with a transmitter. Or both thepatient and the caregiver may wear, carry, or otherwise be associatedwith a transmitter or a receiver or both.

Alternately, or additionally, the patient, the caregiver, or both may beassociated with a bar code, words, QR code, or other visual symbol oridentifier, for example, on an ID badge or bracelet. The 3D motionsensor 120 and/or the computerized monitoring system 130 may note thebarcode, words, QR code, or other visual symbol or identifier, whichcould later be compared to a database to identify the patient orcaregiver, or the 3D motion sensor 120 and/or the computerizedmonitoring system 130 could be given access to a database and configuredto determine the identity of the patient and/or caregiver using thevisual symbol or identifier. A person may be inferred to be a caregiverby identification of clothing such as scrubs, a hospital uniform, a labcoat, etc., in contrast to a hospital gown, nightgown, or streetclothes. Similarly, a person in a hospital gown or nightgown may beinferred to be the patient. In a home or home-like environment, streetclothes may be associated with the caregiver, while in a hospital orinstitutional environment, street clothes may be associated with avisitor.

If desired, the system might not distinguish individuals, or might notseek to distinguish individuals outside of particular circumstances. Forexample, the system may seek to confirm the identity of the patientand/or a caregiver upon system start-up, upon detection of a potentialproblem, upon any event that would trigger the system to send data tothe patient's electronic health record, after specified time intervalsduring monitoring, after random time intervals during monitoring, orcombinations thereof.

The computerized monitoring system 130 may analyze data from 3D motionsensor 120 to determine whether a person is detected, shown as step 140in FIG. 1. The computerized monitoring system may analyze image data,depth data, or both, to evaluate whether any “objects” in the roomappear to be a person. For example, 2D RGB video can be used to identifya person, as can 3D data, such as an IR depth map, ToF data, or stereovision from two or more 2D RGB cameras. A probability-based bounding boxmay be shown around the object that is determined to be a person, usinga predetermined threshold for the confidence that the object is aperson. The bounding box may circumscribe the entire person (a full-bodybounding box), or the bounding box may circumscribe only a portion ofthe person, such as the upper body, lower body, face, etc. One of skillin the art will appreciate that image classifier systems can analyzeimages and predict with a degree of confidence whether an object withinan image fits a certain class, such as “person.” Accordingly, the systemmay look for objects that are likely to be a person with at least 50%confidence, or at least 75% confidence. Using a higher confidenceinterval may lead to the system treating the room as unoccupied anunacceptable number of times when there is, in fact, a person in theroom. The acceptable limits for failing to detect a person will, ofcourse, depend on the purpose of the monitoring and the risk toleranceof the monitoring person or organization. Similarly, the system maycalculate a box around the object likely to be a person based on thedesired confidence that the box circumscribes the entire person, or theentirety of the person in view of the camera, or the desired portion ofthe person, or an acceptable portion of the person or portion of theperson (e.g., most of the upper body, most of the face, etc.). Theconfidence level for circumscribing the entire person or portion of theperson can be set based on the tolerance of the system user for error.For example, if it is critical that a patient not leave bed unassisted,the tolerance for underestimating the extent of the patient may be verylow, because a smaller box would underestimate the risk that the patienthas actually left the bed. Conversely, if the tolerance foroverestimating the extent of the patient is high, the system mayfrequently notify a caregiver of an undesired event when no such eventhas occurred or is likely to occur.

If a person is identified, the computerized monitoring system 130 maydetermine whether the person has crossed an electronic boundary 150. Ahuman-intelligible display as shown in FIGS. 8-9 illustrates this. Ineach of FIGS. 8 and 9, a patient 110 is partially noted by a boundingbox 420. If patient 110 sits up and begins to move out of the bed, thebounding box 110 shifts outside the electronic boundary defined by box510. Returning to FIG. 1, if no person is detected, or if the person (asrepresented by a bounding box 420) has not crossed the virtual barrier510, the computerized monitoring system continues to monitor for aperson and/or a boundary crossing. If a person has crossed the virtualbarrier, an alert is sent to a computerized communication system 160. Analert may be sent only if a specified minimum portion of the boundingbox 420 has crossed the virtual barrier 510.

To assess the patient's position, in addition to or as an alternative tothe use of a bounding box, computerized monitoring system 130 may useskeletal tracking, blob tracking, or other image recognition techniquesto identify one or more tracking points on the patient's body, such aships, shoulders, knees, chin, nose, etc. The patient's position can thenbe analyzed by tracking skeletal segments, or the shape and orientationof a blob, or specified tracking points. For example, the system mayidentify or infer the position of the patient's right knee at a timedesignated as T1, as by the coordinates (x1, y1, z1) of the patient'sright knee in a picture frame. At a later time T2, the patient's rightknee might be at coordinates (x2, y2, z2). Based on this information,motion, speed and direction of movement (or lack of movement) can bederived utilizing the elapsed time and comparing the two 3D coordinates.As opposed to conventional motion sensors, which use captured motion tocontrol a camera, the 3D motion sensor used in the methods and systemsdescribed herein is used to compute the motion. Further, a 3D motionsensor, as opposed to a 2D motion sensor, offers depth sensitivity thatcan help to reduce false alarms (e.g., by identifying rotational orvertical movement, as might occur when a patient rolls to or from oneside of the body), as well as help to isolate the analysis to thepatient and avoid false alarms or false confirmations of repositioningfrom other objects or individuals who might pass in front of or behindthe patient.

Skeletal and/or blob tracking may be used under certain circumstances tosupplement the use of a bounding box. For example, skeletal and/or blobtracking may be used to confirm an initial detection that a virtualbarrier has been crossed. The skeletal and/or blob tracking may beautomatically turned on by the computerized monitoring system 130 ondetection of the crossing of a virtual barrier using bounding boxes, ora human user may have the option to turn on skeletal and/or blobtracking, e.g., if a number of false alerts have been generated for aparticular patient. Alternately, skeletal and/or blob tracking may bepreferentially used for new patients or patients with a history ofwandering off or disrupting medical equipment, and bounding boxes mayturn on if the monitoring does not detect any crossing of the virtualbarrier for a given time period, such as 24 hours or 48 hours. In thisway, the relatively lower data processing and transmission requirementsof the bounding box method can be used when the risk of a problem issomewhat lower, and more data-intensive individual tracking methods canbe used when the risk of a problem is somewhat higher or when thebounding box method is producing an unacceptable number of false alerts.False alerts may be associated with active patients, active rooms (e.g.,many visitors or much activity in the room), or relatively largepatients, whose bounding boxes may take up a relatively greaterproportion of the area within the virtual barrier. Of course, falsealerts using the bounding boxes can also be reduced by modifying theconfidence levels that the object being tracked is a person or that theperson is circumscribed within the bounding box, with higher confidencelevels tending to cause fewer false alerts, or by requiring a minimumportion of the bounding box, greater than 0%, to cross the virtualbarrier before an alarm is sent.

If more than one detection technique or system is used (skeletaltracking, blob tracking and/or bounding box tracking), an alert may besent if any method detects that the virtual barrier has been touched orcrossed, or an alert may be sent only if two or more of the methodsagree that the virtual barrier has been touched or crossed, or an alertmay specify which method(s) detected the virtual barrier has beentouched or crossed and/or whether an alternate method could not confirmthat the virtual barrier has been touched or crossed and, if desired,which alternative method could not make the confirmation.

If a person or the specified minimum portion of the bounding boxrepresenting the person has crossed the virtual boundary, computerizedmonitoring system 130 may send an alert to computerized communicationsystem 160. On receiving an alert from computerized monitoring system130, computerized communication system 160 may send a human-intelligiblesignal or request for attention. For example, computerized communicationsystem 160 may send an alert to an amplifying speaker, publicannouncement system, television, monitor, mobile phone, computer, pager,or other display device in a patient's room. The alert, which could beaudible or visible or both, may request that the patient return to theirprior position. For example, if the virtual barrier is set to detectwhen a patient is trying to get out of bed, the alert may request thatthe patient return to bed and wait for assistance. The alert could betext, sound, or video, or could consist of flashing lights in the roomor on a display, or another visible change in the patient's room, suchas a change in the color of a border of a monitor or television, or achange in the brightness or color of the light in the room. The alertcould take the form of an automated phone call, voice mail message,e-mail message, SMS message, or the like. Alerts to the patient may bedisabled, for example, if the patient is known or believed to be unableor unwilling to respond. For example, if a patient has a history ofseizures or other involuntary movement, an alert to the patient may beunhelpful.

In addition to or instead of alerting the patient, computerizedcommunication system 160 may alert one or more caregivers 100A. As withalerts intended for a patient, alerts for a caregiver could be audibleor visible or both, and may include text alerts, instructions, or othersignals that something is amiss, e.g., flashing lights, color schemes,etc. An alert for a caregiver may be sent to the patient's room, or maybe sent to a device carried by the caregiver, such as a cell phone orpager, or may be sent to a nursing station or dispatch center. An alertfor a caregiver may be sent to a primary caregiver, and, if no change isdetected within a predetermined response time, an alert may be sent toone or more additional caregivers. Alternately, an alert may be sent totwo or more caregivers at the outset. Alerts may also be sent to otherswho might not have primary responsibility for the care of the patient,such as family members or guardians. Alerts, possibly including the 3Dmotion sensor data in the time period before the alert and/or anyresponse(s) to the alert, may be recorded, for example, in database 170.Database 170 may include, or may provide information to, or may beaccessible by, an Electronic Health Record (EHR) for one or morepatients. In this way, alerts and responses may be recordedautomatically in an EHR without caregiver input. Exemplary responses toan alert may include a human operator cancelation of the alert (e.g.,based on a caregiver or centralized monitoring station attendantconfirmation that the patient is safe or has returned within the virtualboundary), a system determination that the patient has returned withinthe virtual boundary, or a system determination that a second person hasentered the virtual boundary (presumably a caregiver or cliniciantending to the patient). If desired, the computerized monitoring systemmay identify individuals before clearing an alert, to confirm that thesecond person crossing the virtual boundary is a caregiver or clinician,or even to document the identity of the individual responding to thealert.

A confirmation that an alert has been resolved—because the patientreturned within the virtual barrier or because a caregiver or clinicianhas response—may be communicated to a patient, a caregiver and/orothers, in any of the modes and manners described for alerts. Aconfirmation that an alert was resolved may also be recorded in database170, an EHR, or in any other desired file or storage location.

Computerized monitoring system 130 and/or computerized communicationsystem 160, shown in FIG. 2 as combined computerized monitoring andcommunication systems 210A, 210B, and 210C, may also be in communicationwith a centralized monitoring station 200. A centralized monitoringstation 200 may be used with a single 3D motion sensor 120 for a singlepatient. For example, centralized monitoring station 200 may include adisplay in a home of a family member or guardian of patient 110. Asshown in FIG. 2, a plurality of 3D motion sensors 120A, 120B, and 120Cmay monitor a plurality of patients, 110A, 110B, and 110C, respectively.The 3D motion sensors 120A, 120B, and 120C may be monitored by distinctcomputerized monitoring and communication systems 210A, 210B, and 210C,respectively. Alternately, 3D motion sensors 120A, 120B, and 120C couldeach send 3D motion and/or sound data to a single computerizedmonitoring system 130 or to a single combined computerized monitoringand communication system.

The computerized monitoring system 130 and/or computerized monitoringand communication systems 210A, 210B, and 210C may send filtered orunfiltered data, such as images and/or a live video feed, with orwithout sound, from 3D motion sensors 120A, 120B, and 120C tocentralized monitoring station 200. The 3D motion sensor data may bereceived and displayed as human-intelligible images on a centralizedmonitoring primary display 230, which may be a single display monitor ora series or grid of two or more display monitors. As mentioned above,the computerized monitoring system and/or the centralized monitoringstation may apply filters before the 3D motion sensor data is displayed,for example, to blur or pixelate the face or body of the patient, toprotect patient privacy. In addition, video and/or sound, if sound isprovided, can be turned off at any node, including centralizedmonitoring primary display 230 and directly at the 3D motion sensor, toprotect patient privacy, e.g., while the patient is receiving visitors,bathing, changing clothes, etc. If a large number of patients are beingmonitored at the same time, the centralized monitoring primary display230 may be enlarged so that it can aggregate multiple telemetry feeds,or more than one centralized monitoring station primary display 230could be used. Regardless of whether the data is filtered or unfiltered,it may still be processed by the computerized monitoring system 130, acomputerized monitoring and communication system (e.g., 210A, 210B, or210C) and/or the centralized monitoring station 200 to render the dataas a human-intelligible visual image or series of images (e.g., video).

When the computerized communication system receives an alert, thecomputerized communication system may send the alert to the centralizedmonitoring station 200. At step 240, on receipt of a determination fromthe computerized monitoring system 130 and/or an alert from thecomputerized communication system 160 for a particular 3D motion sensor,the display from that sensor may be moved from centralized monitoringstation primary display 230 to centralized monitoring station alertdisplay 250 or duplicated on centralized monitoring station alertdisplay 250. Centralized monitoring station alert display 250 may be asubset of the display or displays of centralized monitoring stationprimary display 230, or may be a distinct display or series of displays.If live video is available but was not being displayed on centralizedmonitoring station primary display 230, the live video may be displayedon centralized monitoring station alert display 250 after an alert isreceived. Centralized monitoring station alert display 250, or anattendant there, may analyze the video feed to determine what ishappening in the patient's room. If a caregiver has arrived and istending to the patient or otherwise correcting the cause of the alert,the centralized monitoring station alert display 250 or attendant mayclear the alert. If an alert has been sent to a caregiver and noresponse is detected or received, centralized monitoring station alertdisplay 250 or an attendant may notify an alternate or back-up caregiverthat the patient needs assistance. Alerts and any actions taken orresponses received or observed at centralized monitoring station 200 maybe recorded, for example, to database 170.

The centralized monitoring station primary display 230 may routinelydisplay live video for monitored patients. An attendant at thecentralized monitoring station primary display 230 can use the livevideo feed to detect other problems, such as a patient fall, a patientgesture that he or she needs assistance, an unauthorized person hasentered the patient's room, etc.

The various system components and/or method steps may be situated and/orperformed remotely from one another. So long as the components cantransfer data and perform the functions described, the components or anysubcombination of components could be located together, even, in someaspects, in a singular physical housing. Alternately, the components orany subcombination of components could be remote from one another,either in different rooms, different floors of a building, differentbuildings, different cities, or even different countries or continents.The centralized monitoring station 200, for example, may reside at anursing station on the same floor or on a different floor of the samebuilding as the 3D motion sensor, or could be in a regional center thatreceives telemetry from a plurality of 3D motion sensors in differentrooms, buildings, or even cities, and possibly in a variety of patientenvironments. That is, a computerized monitoring system, computerizedcommunication system and/or centralized monitoring station may processdata from 3D motion sensors in hospitals, outpatient centers, assistedliving facilities, and/or private homes, or may be specific, e.g., to aparticular patient or healthcare organization (such as a hospital orhospital network).

The computerized monitoring system and/or centralized monitoring stationmay allow a user to configure a virtual barrier or boundary around apatient or a location where the patient spends time, such as a bed,chair, chaise, etc. FIG. 3 shows an exemplary display 300 of visualtelemetry data for multiple patients 110A, 110B, and 110C, insimultaneous views 310A, 310B, and 310C, respectively, as might beconfigured on centralized monitoring station primary display 230. Asshown, views 310A, 310B, and 310C appear on a split screen, however,different views could also be shown on separate displays. In addition toshowing patients 110A, 110B, and 110C, display 300 shows bounding boxes320A, 320B, and 320C for each patient. It will be appreciated thatalthough FIGS. 3-9 show a skeleton figure within each of the boundingboxes 320A, 320B, and 320C, the bounding boxes could be used without anyadditional tracking means, or any other suitable means of tracking thepatient's body position could be used, including, without limitation,blob tracking, object tracking, or other image recognition techniques toidentify one or more specific tracking points on the patient's body,such as the patients hip(s), shoulder(s), knee(s), chin, nose, etc. Thebounding boxes 320A, 320B and 320C are of slightly different shapes andorientations, although each of the bounding boxes circumscribes most orall of the person being monitored. The bounding boxes need not encompassthe entire person, and the bounding boxes need not have the same extentsettings (e.g., full-body, upper-body, lower-body, face, etc.) for eachperson being monitored. As such, the bounding boxes 320A, 320B, 320Ccould have more pronounced differences in shape, size and orientationthan shown in FIG. 3. In addition, view 310C shows a pop-up menu 330,which may present configuration options for view 310C or options forresponding to an alarm associated with monitored patient 110C or both.

FIG. 4 shows an exemplary display 300 of visual telemetry data for asingle patient 110, with bounding box 420 and menu 400. FIG. 5 shows thesame exemplary display 300 as FIG. 4 after a user has selected a menuoption to define and/or confirm a virtual barrier 510. The virtualbarrier 510 may be automatically generated by the computerizedmonitoring system 130. For example, computerized monitoring system 130may define a virtual barrier 510 by outlining the perimeter orsilhouette of an object, such as a bed or chair, or a line, for exampleacross the threshold of a door. Virtual barrier 510 is shown asrectangular, but virtual barrier could be any closed figure, such as anoval, circle, square, hexagon, etc., or could be a line, arc, or openfigure, such as a zig-zag line or semi-circle.

Analysis of image data may be limited to the virtual barrier 510 andbounding box 420 to reduce the processing capacity required to performthe analysis. If image data is transferred between remote systemcomponents, only data from the virtual barrier 510 and bounding box 410may be routinely transferred, to reduce bandwidth and storage capacitiesrequired for the system's operation. In some aspects, the system may beconfigurable to analyze, transmit, and/or store all data within thefield of view of the 3D motion sensor, either routinely or on theoccurrence of a specified event, such as an alert.

As shown in FIG. 6, the virtual barrier 510 may have a third dimensionof depth, e.g., be defined as a volume. The depth of the virtual barrier510 may be automatically generated by the computerized monitoring system130. By selecting a configuration option from menu 400, a user may alteror reset the depth that defines virtual barrier 510 using a pop-up menu600. Alternately, the extent, placement and/or depth of virtual barrier510 may be determined entirely by a system user, such as by enteringcoordinates or distances, as shown in pop-up menu 600 in FIG. 6, or byproviding selection tools like drag-and-drop and pull-to-expand boxes orother shapes or tools.

FIG. 7 shows a configured virtual barrier 510 overlaid on visualtelemetry for a monitored individual 110, surrounded by bounding box420. FIG. 8 shows additional configuration options 800 from menu 400,allowing a user to select whether to display video telemetry (“VIDEOFEED”), audio telemetry (“AUDIO FEED”), or both. FIG. 9 shows patient110 moving to the side of the bed. As the patient moves, so doesbounding box 420. When bounding box 420 moves beyond virtual barrier510, as shown in FIG. 9, computerized monitoring system 130 detects thecrossing of the virtual barrier 510 by the bounding box 420 and issuesan alert.

Computerized monitoring system 130 may determine that the bounding box420 has crossed the virtual barrier 510 only if a minimum portion of thebounding box 420 has crossed the virtual barrier 510. Portions of thebounding box 420 can be measured, for example, as a percentage of thepixel area of the bounding box, a minimum number of pixels within thebounding box, or a percentage of the volume of the bounding box.Measuring a percentage of the volume of the bounding box requires usinga depth parameter for the bounding box, however, even if a depthparameter is used, a 3D bounding box may still be evaluated for crossingthe virtual barrier based on pixel area or number of pixels. The portionof the bounding box 420 that must cross the virtual barrier 510 totrigger an alarm can be set at any desired level, such as at least 0% ofthe pixel area or volume of the bounding box (zero percent, e.g., if thebounding box touches but does not cross the virtual barrier), at least10% of the pixel area or volume of the bounding box, at least 20% of thepixel area or volume of the bounding box, or at least 30% of the pixelarea or volume of the bounding box. It will be appreciated that settingthe threshold for an alarm to a smaller percentage or number of pixelswill result in alarms based on less movement by the person beingmonitored, with a potentially higher number of alarms, and setting thethreshold for an alarm to a larger percentage or number of pixels willresult in alarms only with greater movement by the person beingmonitored toward the virtual barrier, with a potentially lower number ofalarms. In this way, setting a minimum portion of the bounding box thatmust cross the virtual barrier to trigger an alarm is a way to adjustthe sensitivity of the detection method, and potentially to reduce falsealarms (increasing the minimum portion of the bounding box that mustcross the virtual barrier to trigger an alarm) or to trigger an alarmmore quickly upon detection of possible intent to cross the virtualbarrier (decreasing the minimum portion of the bounding box that mustcross the virtual barrier to trigger an alarm).

The systems, methods, and media described may be operated in anexemplary computing environment 1000 as shown in FIG. 10. Exemplarycomputing environment 1000 includes at least one computing device in theform of a control sever 1030. Components of control server 1030 mayinclude, without limitation a processing unit, internal system memory,and a suitable system bus for coupling various system components,including database cluster 1020, with the control server 1030. Thesystem bus may be any of several types of bus structures, including amemory bus or memory controller, a peripheral bus, and a local bus,using any of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronic Standards Association (VESA) local bus, andPeripheral Component Interconnect (PCI) bus.

The control server 1030 typically includes therein, or has access to, avariety of computer-readable media, for instance, database cluster 1020.Computer-readable media can be any available media that may be accessedby control server 1030, and includes volatile and nonvolatile media, aswell as removable and non-removable media. By way of example, and notlimitation, computer-readable media may include computer-storage mediaand communication media. Computer-storage media may include, withoutlimitation, volatile and nonvolatile media, as well as removable andnon-removable media implemented in any method or technology for storageof information, such as computer readable instructions, data structures,program modules, or other data. In this regard, computer-storage mediamay include, but is not limited to, RAM, ROM, EEPROM, flash memory orother memory technology, CD-ROM, digital versatile disks (DVDs) or otheroptical disk storage, magnetic cassettes, magnetic tape, magnetic diskstorage, or other magnetic storage device, or any other medium which canbe used to store the desired information and which may be accessed bythe control server 1030. Computer-storage media may exclude signals perse. Computer-readable media may exclude signals per se.

Communication media typically embodies computer readable instructions,data structures, program modules, or other data in a modulated datasignal, such as a carrier wave or other transport mechanism, and mayinclude any information delivery media. As used herein, the term“modulated data signal” refers to a signal that has one or more of itsattributes set or changed in such a manner as to encode information inthe signal. By way of example, and not limitation, communication mediaincludes wired media such as a wired network or direct-wired connection,and wireless media such as acoustic, RF, infrared, and other wirelessmedia. Combinations of any of the above also may be included within thescope of computer-readable media. The computer-storage media discussedabove and illustrated in FIG. 10, including database cluster 1020,provide storage of computer readable instructions, data structures,program modules, and other data for the control server 1030.

The control server 1030 may operate in a computer network 1010 usinglogical connections to one or more remote computers 1040. Remotecomputers 1040 may be located at a variety of locations in a medical orresearch environment, for example, but not limited to, clinicallaboratories (e.g., molecular diagnostic laboratories), hospitals andother inpatient settings, veterinary environments, ambulatory settings,medical billing and financial offices, hospital administration settings,home health care environments, and clinicians' offices and theclinician's home or the patient's own home or over the Internet.Clinicians may include, but are not limited to, a treating physician orphysicians, specialists such as surgeons, radiologists, cardiologists,and oncologists, emergency medical technicians, physicians' assistants,nurse practitioners, nurses, nurses' aides, pharmacists, dieticians,microbiologists, laboratory experts, laboratory technologists, geneticcounselors, researchers, veterinarians, students, and the like. Theremote computers 1040 may also be physically located in non-traditionalmedical care environments so that the entire health care community maybe capable of integration on the network. The remote computers 1040 maybe personal computers, servers, routers, network PCs, peer devices,other common network nodes, or the like, and may include some or all ofthe elements described above in relation to the control server 1030. Thedevices can be personal digital assistants or other like devices. Asdescribed above, one or more of the remote computers 1040 may bespecifically designed and/or configured to perform certain functions inrelation to the systems and methods disclosed, distinguishing thosedevices from general purpose computers.

Exemplary computer networks 1010 may include, without limitation, localarea networks (LANs) and/or wide area networks (WANs). Such networkingenvironments are commonplace in offices, enterprise-wide computernetworks, intranets, and the Internet. When utilized in a WAN networkingenvironment, the control server 1030 may include a modem or other meansfor establishing communications over the WAN, such as the Internet. In anetworked environment, program modules or portions thereof may be storedand/or executed on the control server 1030, in the database cluster1020, or on any of the remote computers 1040. For example, and not byway of limitation, various application programs and/or data may resideon the memory associated with any one or more of the remote computers1040. It will be appreciated by those of ordinary skill in the art thatthe network connections shown are exemplary and other means ofestablishing a communications link between the computers (e.g., controlserver 1030 and remote computers 1040) may be utilized.

In operation, a user may enter commands and information into the controlserver 1030 or convey the commands and information to the control server1030 via one or more of the remote computers 1040 through input devices,such as a keyboard, a pointing device (commonly referred to as a mouse),a trackball, a touchscreen or a touch pad. Other input devices mayinclude, without limitation, microphones, satellite dishes, scanners, orthe like. Commands and information may also be sent directly from aremote healthcare device to the control server 1030. In addition to amonitor, the control server 1030 and/or remote computers 1040 mayinclude other peripheral output devices, such as speakers and a printer.

Many other internal components of the control server 1030 and the remotecomputers 1040 are not shown because such components and theirinterconnection are well known. Accordingly, additional detailsconcerning the internal construction of the control server 1030 and theremote computers 1040 are not further disclosed herein.

Methods and systems of embodiments of the present disclosure may beimplemented in a WINDOWS or LINUX operating system, operating inconjunction with an Internet-based delivery system. One of ordinaryskill in the art will recognize that the described methods and systemscan be implemented in any alternate operating system suitable forsupporting the disclosed processing and communications. As contemplated,the methods and systems of embodiments of the present disclosure mayalso be implemented on a stand-alone desktop, personal computer,cellular phone, smart phone, PDA, or any other computing device used ina healthcare environment or any of a number of other locations.Nonetheless, when networked and/or programmed as described herein, thesystem does more than the individual, generic devices could do.

It will be appreciated by one of skill in the art that the methodsdisclosed may be performed by one or more computing devices executinginstructions embodied on computer-readable media. The instructions maycause the one or more computers to perform the method steps disclosedwhen executed by one or more processors associated with the one or morecomputers. Such media may be embodied in hardware, firmware, computerstorage media, computer-readable media, or other devices or technologiesnow known or later arising.

Computer-readable media embodying one or more computer executableinstructions may include a data acquisition module. The data acquisitionmodule may acquire data from one or more 3D motion sensors 120. The dataacquisition module may receive data sent to it by one or more devices,or may request or retrieve data from one or more devices. The dataacquisition module may further acquire data from one or more databasesand/or electronic files available to the system, such as an EHR system,a database containing photographs or other identifiers for patientsand/or caregivers, or any other system or file containing usefulinformation.

A virtual barrier module may define and/or provide a user interface fordefining, altering, and/or confirming a virtual barrier. The virtualbarrier module may use the virtual barrier to create one or more subsetsof data acquired by the data acquisition module. For example, a subsetof data limited from the entire field of view of a 3D motion sensor maybe generated for only data from within a virtual barrier or within aspecified distance of a virtual barrier. Analysis of the data may belimited to the subset of data, so as to conserve processing capacity inthe system by not analyzing data from portions of the field of view thatare unlikely to be relevant to whether or not the virtual barrier hasbeen crossed. Similarly, subsets of data may be defined forcommunication and/or storage purposes, to conserve communicationsbandwidth and/or storage capacity requirements for the system.

A person identification module may be configured to analyze dataacquired by the data acquisition module, and/or subsets of data acquiredby the data acquisition module. The person identification module mayclassify objects in images from one or more 3D motion sensors, andidentify objects that meet a minimum confidence threshold forclassification as a person. The person identification module may furtheruse facial recognition, ID tokens (including RFID or other signalingtokens), ID markers (including bar codes, name tags, and the like), orother identifiers to identify the patient as an individual. If the dataacquisition module has accessed patient files or other sources ofidentifiers for patients, caregivers or clinicians, the personidentification module may compare data from the 3D motion sensors orother sensors (e.g., RFID readers) with other information to identifythe person individually, e.g., by name or patient number. Alternately,facial recognition or other biometric data may be used to distinguishone person from other people who may be in or come in and out of theimages, such as visitors, caregivers, clinicians, etc., withoutidentifying the person by name. Identifying or distinguishing a personindividually may be helpful if more than one person is present in theimage, e.g., to prevent sending alarms when a visitor crosses a virtualbarrier intended for a patient, or when a patient crosses a virtualbarrier intended for a caregiver (e.g., a boundary that the caregivershould not cross without washing his or her hands).

An alert module may monitor the position of any bounding boxes relativeto the virtual barrier. The alert module may generate an alert if thebounding box touches or crosses the virtual barrier, or if a specifiedminimum portion of the bounding box crosses the virtual barrier. If thebounding box, the virtual barrier, or both is an open figure or includesirregular boundaries (e.g., boundaries which are not straight lines orsingle-radius curves), an alert may be generated if any portion of thebounding box touches or crosses any portion of the virtual barrier, orif a specified minimum portion of the bounding box crosses any portionof the virtual barrier or a specified minimum portion of the virtualbarrier. As with the bounding boxes, portions of the virtual barriercould be measured as a minimum number of pixels, a minimum percent ofthe pixel area within the virtual barrier, or a minimum percent of thevolume of a 3D virtual barrier, and the minimum portion of the virtualbarrier could be 0% or any percentage greater than 0% and less than orequal to 100%. An alert may be sent only if the entire bounding boxcrosses the virtual barrier. An initial alert may be sent if a portionof the bounding box touches or crosses any portion of the virtualbarrier, and another alert may be sent if the entire bounding boxcrosses the virtual barrier. Additional alerts could be sent asprogressively larger portions of the bounding box crosses the virtualbarrier. For example, if the monitoring is to detect a patientattempting to get out of bed without assistance, as for fall prevention,an alert may be sent if the bounding box touches any portion of thevirtual barrier. This alert might not trigger an immediate response, forexample, if a caregiver is busy or if the person being monitored isactive. However, an alert indicating that 10% of the bounding box hascrossed the virtual barrier may be of higher priority, because it ismore likely that the patient is continuing to move across the virtualbarrier. Similarly, an alert indicating that 20% or 30% or more of thebounding box has crossed the virtual barrier may signal that animmediate response is needed to prevent the patient from leaving the bedunassisted.

A communication module may be configured to receive alerts and/orconfirmations from the alert module, and to select and transmit anappropriate message to a patient, a caregiver, an alternate caregiver,other human users, a centralized monitoring station, and/or others. Thecommunication module may select a mode of communication, e.g., anautomated phone call, an e-mail, a text display in a patient's room,etc., based on user preferences, default settings, and/or the nature ofthe message to be communicated. The communication module may select amessage based on the nature of the communication (e.g., whether thebounding box has partially or completely crossed the virtual barrier,whether the determination based on the bounding box has been confirmedby alternative analyses and/or human confirmation), and may furtherselect a language for delivering the message. Different modes ofcommunication, different message content and/or different languages maybe selected for different alert recipients, or the same alert orconfirmation may be sent to all recipients, using the same or differentmodes of communication for different recipients, if there is more thanone recipient.

A central monitoring module may aggregate data from the monitoring ofmultiple patients. Image data and/or video, if available, may bedisplayed on a primary display, rendered as human-intelligible images.The central monitoring module may move a display of data related to apatient to an alert display, or duplicate a display of data related to apatient on an alert display, upon receiving an alert for that patient.The central monitoring module may move a display of data related to apatient to a primary display, or may remove a display of data related toa patient from an alert display, after receiving a response to an alert.The central monitoring module may be configured to permit a humanattendant using the central monitoring module to access thecommunication module to send a message to one or more recipientsregarding an alert, a response to an alert, a lack of response to analert, and/or confirmation that an alert condition has been corrected.Messages sent via the central monitoring module may be pre-recorded orpre-determined (e.g., selected from a menu of available messages) or maybe recorded, typed, or otherwise entered by the human attendant via thecentral monitoring module and communication module.

A recordation module may store data related to alerts and/orconfirmations, including any received response (e.g., a response enteredinto the system by a human user), observed response, or the apparentlack of a response to any alert. The data may be stored in a databaseassociated with the system, or in any other desired electronic file orstorage location. In some embodiments, the recordation module may storedata for a particular patient in an EHR, case report form, or othermedical recordkeeping file. In some embodiments, the recordation modulemay store data in a database or file accessible to an EHR system and/orother systems. In some embodiments, the recordation module may store alldata acquired for a particular patient, or only data regarding alertsand/or confirmations, or only data for a designated timeframe.

All steps and flowcharts described herein, including in the attachedfigures, are meant to be illustrative. It should be understood thatother steps may be used with the illustrated steps, and, further, thatsome steps may be useful without the practice of other steps included inthe figures. The illustrated sequence of steps is also exemplary, and,unless described otherwise, the steps may be performed in differentsequences, combinations, and/or subcombinations.

From the foregoing, it will be seen that this invention is one welladapted to attain all the ends and objects hereinabove set forthtogether with other advantages which are obvious and which are inherentto the structure.

It will be understood that certain features and subcombinations are ofutility and may be employed without reference to other features andsubcombinations. This is contemplated by and is within the scope of theclaims.

Since many possible embodiments may be made of the invention withoutdeparting from the scope thereof, it is to be understood that all matterherein set forth or shown in the accompanying drawings is to beinterpreted as illustrative and not in a limiting sense.

What is claimed is:
 1. A computerized method of identifying the crossingof a virtual barrier, the method being performed by a computerizedmonitoring system and comprising: receiving image data for a room fromone or more 3D motion sensors; configuring a virtual barrier within theroom; classifying an object within the room as a person; creating abounding box to circumscribe at least a portion of the person accordingto a predetermined confidence level that is based on a risk tolerancelevel, wherein the predetermined confidence level indicates a thresholdconfidence in classifying the object within the room as a person tocreate the bounding box; and monitoring a position of the bounding boxover time and relative to the virtual barrier.
 2. The computerizedmethod of claim 1, wherein the risk tolerance level is based on theperson being monitored.
 3. The computerized method of claim 1, whereinthe risk tolerance level is based on an activity being monitored.
 4. Thecomputerized method of claim 3, wherein the activity being monitored isa fall risk.
 5. The computerized method of claim 3, wherein the activitybeing monitored is leaving a bed unassisted.
 6. The computerized methodof claim 1, wherein the risk tolerance level is based on the personbeing monitored and an activity being monitored.
 7. The computerizedmethod of claim 1, wherein a high risk tolerance level is associatedwith a high predetermined confidence level.
 8. The computerized methodof claim 1, wherein the bounding box is configured to circumscribe anentire body of the person.
 9. A system for identifying the crossing of avirtual barrier, the system comprising: a computerized monitoring systemin communication with one or more 3D motion sensors; and a computerizedcommunication system; wherein the computerized monitoring system isconfigured to: receive image data from the one or more 3D motionsensors; configure a virtual barrier; analyze the image data to detect aperson and create a bounding box around at least a portion of the personin the image data according to a predetermined confidence level that isbased on a risk tolerance level, wherein the predetermined confidencelevel indicates a threshold confidence in detecting the person to createthe bounding box; and send an alert to the computerized communicationsystem upon detecting that the bounding box touched or crossed thevirtual barrier.
 10. The system of claim 9 further comprising a userinput device, wherein the computerized monitoring system configures thevirtual barrier based on user input from the user input device.
 11. Thesystem of claim 9, wherein the computerized monitoring systemautomatically configures the virtual barrier.
 12. The system of claim 9,wherein upon detecting that the bounding box has touched or crossed thevirtual barrier, the computerized monitoring system utilizes skeletaltracking, blob tracking, or combinations thereof to confirm the touchingor crossing of the virtual barrier.
 13. The system of claim 9, whereinthe computerized monitoring system utilizes biometrics to identify theperson.
 14. The system of claim 9, wherein the risk tolerance level isbased on the person being monitored and an activity being monitored. 15.The system of claim 14, wherein the activity to be monitored is leavinga bed unassisted.
 16. The system of claim 9, further comprising acentralized monitoring station configured to display images based on theimage data from the one or more 3D motion sensors.
 17. Non-transitorycomputer-readable media having embodied thereon instructions that, whenexecuted by a computer processor, cause the processor to: receive imagedata for a room from one or more 3D motion sensors; configure a virtualbarrier within the room; classify an object within the room as a person;create a bounding box to circumscribe at least a portion of the personaccording to a predetermined confidence level that is based on a risktolerance level, wherein the predetermined confidence level indicates athreshold confidence in classifying the object within the room as aperson to create the bounding box; and monitor a position of thebounding box over time and relative to the virtual barrier.
 18. Thenon-transitory computer-readable media of claim 17, wherein the risktolerance level is based on the person being monitored and an activitybeing monitored and wherein a higher risk tolerance level is associatedwith a higher confidence level.
 19. The non-transitory computer-readablemedia of claim 17, wherein the instructions further cause the processorto send an alert to a computerized communication system upon determiningthat the bounding box crossed the virtual barrier.
 20. Thenon-transitory computer-readable media of claim 19, wherein theinstructions further cause the processor to send another alert to thecomputerized communication system upon determining that all of thebounding box crossed the virtual barrier.